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IMAGE PROCESSING ^"^^^^ ^ 

HiiM|.f^f€iieeii Kaasan 

TECHNICAL FIELD 
The present invention generally refers to image processing, and in particular 
5 to methods and systems for encoding and decoding images. 

BACKGROUND 

Presentation and rendering of images and graphics on data processing 
systems and user terminals, such as computers, and in particular on mobile 
10 terminals have increased tremendously the last years. For example, three- 

dimensional (3D) graphics and images have a number of appealing 
applications on such terminals, including games, 3D maps and messaging, 
screen savers and man-machine interfaces. 

15 A 3D graphics rendering process typically comprises three, sub-stages. 

Briefly, a first stage, the application stage, creates several triangles. The 
comers of these triangles are transformed, projected and lit in a second 
stage, the geometry stage. In a third stage, the rasteri^tion stage, images, 
often denoted textures, can be "glued^ onto the triangles, increasing the 

2 0 realism of the rendered image. The third stage also performs sorting using a 

2-buffer. 

However, rendering of images and t^sstures, and in particxilar 3D images and 
graphics, is a computationally expensive task in terms of memoiy bandwidth 
25 and processing power required for the graphic systems. For example, 

- ^ > , textures are costly both in terms of memory, the textures must be placed on 

♦ ^ • * 

^ fast on-chip memory, and in terms of memory bandwidth, a texture can be 
accessed several times to draw a single pixel. 



0-5 In order to reduce the bandwidth and processing power requirements, an 

/ image (texture) encoding method or system is typically employed. Such an 
encoding system should result in more efficient usage of expensive on-chip 
memory and lower memoiy bandwidth during rendering and, thus, in lower 



r » 
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power consumption and/or faster rendering. This reductioiiW^4t|dfeig^d§BK»4th 
and processing power reqijirements i$ particularly impoitant for thin clients, 
such as mobile units and telephones, with a smajll amount of memoryj little 
memory bandwidth and limited power (powered by batteries). 
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Delp and Mitchell [1] describe a simple scheme, called block truncation 
coding (BTC), for image encoding. Their scheme encodes gray scale images 
by considering a block of 4 pi^cels x 4 pixels at a time. For each such block, 
two 8-bit gray scale values arc generated and each pix:el in the block then 
uses a single bit to index one of these gray scales. This results in a 
compression rate of 2 bits per pixel (bpp). However, BTC suffers from 
production of artifacts, above all in regions around edges and in low contrast 
eareas containing a sloping gray level. Furthermore, edges in a gray scale 
image processed by BTC have a tendency to be ragged, 

A simple extension of BTC, called color cell compression (CCC), was 
presented by Campbell et al- [2]. Instead of using two 8-bit gray scale values 
for each image block, two 8-bit values are employed as indices into a color 
palette. This palette comprises 256 colors represented by 8 bits for each of 
the R, G and B component. An 8-bit index then points at a (24-bit) color in 
the palette. This allows for compression of images at 2 bpp. However, this 
does require a memory lookup in the palette. In addition, the palette is 
restricted in $i:5e. The CCC scheme also introduces large color "jaggies** and 
poorly encodes the case where more than two colors exist in an image block. 

In a patent description [3], lourcha et al. disclose a texture compression 
scheme called S3TC (S3 Texture Compression) or DXTC (DirectX Texture 
Compression), that can be seen as an extension of CCC, An image is 
decomposed into a number of image blocks of 4 pixels x 4 pixels. Each such 
image block is encoded into a bit sequence of 64 bits, thus resulting in a 
compression rate of 4 bpp. The 64-bit sequence comprises two basic colors 
or color codewords (16 bits each) and a 32-bit sequence of 2-bit indices, one 
index for each pixel in the image block* During decoding, a color palette of 
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four colors is generated. The first two RGB (red, green and bl\j.e) colors of the 
palette correspond to the two basic colors (codewords). The two additional 
colors, situated between the basic colors in the RGB space, are then 
interpolated therefrom. Each 2-bit index then identifies, for each pixel, one of 
the four colors of the palette to use for that pixel. 

Although, the S3TC scheme works fairly well for computer terniinals, it is 
not well adapted for mobile units and other thin clients. Such mobile units 
typically only have meinory busses of 16 or 32 bits at best. Thus, at least 
two, and possibly up to four, memoiy accesses are rquired to read out the 
64-bit compressed version of an image, block, if S3TC is implemented in a 
mobile unit- In addition, during the interpolation of the two additional colors 
of the color palette, multiplication by 1/3 and 2/3 is performed, which is not 
ideal in hardware. The compression using S3TC is also relatively time 
consuming, at least on a mobile terminal. 

Akenine-Moller and Strom [4] have developed a variant of S3TC, called 
POOMA, which is specifically targeted for mobile phones. In POOMA, an 
image block of 3 pixels x 2 pixels is encoded into 32 bits, giving 5,33 bpp. 
The encoded 32-bit representation of an image block is adapted for the 
memory busses of mobile phones, which typically are 32 bits at best. Thus, a 
pbcel can be rendered using only one memory access compared to two 
accesses for S3TC. POOMA uses two base colors but only one additional 
color interpolated between the base colors, resulting in a color palette of 
three colors. 



A major disadvantage of POOMA is the block size of 3 x 2 pixels. As a result, 
calculation of the block address for a particular pixel or tcxel (texture 
element) requires division by 3, which is not ideal for hardware design. 
Furthermore, the widths and heights of textures (images] in graphics are 
typically always a power of two, which means that a block width of 3 is 
inconvenient. As for S3TC, the encoding using POOMA is relatively time 
consuming, in particular when implemented on a mobile terminal- 
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Penny [5] discloees an image-encoding scheme used in the MBX graphics 
hardware platform for mobile phones. This scheme uses two low-resolution 
images, where one image is usually a low-pass filtered version of the original 
image. During decoding a bilinear magnification (upscaUng) of these two 
images is created. Each pixel also stores a blend factor between these two 
upscaled images- 64 bits are used for encoding each image block and a 
compression rate of 2 bpp and 4 bpp is described- Infoimation from 
neighboring image blocks is needed, which complicates decoding. 

SUMMARY 

The present invention overcomes these and other drawbacks of the prior art 
arrangements- 



15 



It is a general object of the present invention to provide an efficient icnage 
processing. 



It is another object of the invention to provide an efficient image encoding 
and image decoding. 



20 



Yet another object of the invention is to provide image encoding and 
decoding adapted for usage in thin clients with low memory and memory 
bandwidth capacity. 



25 



A further object of the invention is provide image encoding and decoding 
adapted for three-dimensional (3D) graphics and images. 



These and other objects are met by the invention as defined by the 
accompanying patent claims. 



Briefly, the present invention involves image processing in the form of 
encoding (compressing) ah image and decoding (decompressing) an encoded 
(compressed) image. 



'03 12/19 14:03 FAX +46 18 153050 AROS P^iCENT AB 

+46 18 153050 



10 



15 



20 



P ATENTVERKET ® 0 0 7 

Ink. t Patent- och r6g,verk8t 
2003 -Vi- 1 9 



According to the invention, an image to be encoded is decomposed into a 
number of image blocks comprising m\atiplc image elements (pixels or 
texture elements, texels). An image block preferably comprises eight image 
elements and has a size of 2^x2^ image elements, where m=3-n and n-0, 1, 
2, 3, Each image element in a block is characterized by a color, e.g. a 12-bit 
RGB (red, green, blue) color. The image blocks are then encoded. 

In this lossy block encoding, a color codeword is determined for the image 
block. The color codeword is a representation of the colors of the image 
elements of the image block. A preferred representation is an average value of 
the colors of the image elements in block, quanti^d into 12 bits {4 bits for 
each of the three color components for a RGB color). Thus, one and the same 
color codeword (i.e. color representation) is generated for an image block, i.e. 
for all the image elements of the image block. Thereafter, an intensity 
codeword is provided for the image block- This intensity codeword is a 
representation of a set of multiple intensity modifiers that are used (during 
decoding) for modifying or modulating the intensity of the image elements in 
the image block. Once the intensity codeword is provided, intensity 
representations for the image elements in the image block are selected. Each 
such intensity representation is associated with an intensity modifier from the 
intensity modifier set. In other words, the intensity representation allows 
identificatLon of which intensity modifier from the set to use for a specific 
image element of the block. 



25 



It > 
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The resulting encoded image block then comprises the color codeword, 
preferably 12 bits, the intensity codeword, preferably 4 bits, and a sequence of 
the intensity representations, preferably 8x2=16 bits. The resulting si2© of an 
encoded image block is, thus, only 32 bits and a compression rate of 4 bits per 
pixel (image element) is obtained. This small 32-bit size is well adapted for thin 
clients, such as mobile units and telephones, which typically have memoiy 
busses of 16 or 32 bits. As a result, only one or at worst two memory 
accesses are then needed to read out the encoded image block from a 
memory location. 
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In a preferred embodiment of the invention, the intensity codeword is an 

♦ 

intensity index allowing identification of an intensity modifier set. This index 
could then identify or point to the set in a table or codebook comprising 
several different intensity modifier, sets. Each set preferably comprises four 
(mathematically) complementaiy modifier values- In such a case, the modifier 
sets stored in the table only have to comprise two different intensity modifier 
values each and then the other two (complementary) values of the set could be 
calculated therefrom. In addition, the intensity table preferably comprises 
both sets that include small intensity modifier values, which are adapted for 

• * 

allowing representation of smoothly changing siorfaces, and sets that include 
large intensity modifier values, which are adapted for allowing representation 
of sharp edges. 



15 



20 



During decoding, the encoded image .block(s) that should be decoded is 
identified and fetched from e.g. a memoiy location. Once the correct encoded 
image block is identified an intensity niodifier set is provided. This modifier 

■ I 

set is provided based on the intensity codeword in the encoded image block. 
This set provision is preferably performed by identifying, by means of the 
intensity codeword, an intensity modifier set from an intensity table 
comprising multiple modifier sets. 



25 
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m 



Thereafter, a color representationVis generated for at least one of the image 
elements of the image block. This color generation is performed based on the 
color codeword in the encoded block representation. The intensity modifier to 
use for the image element that shotlld be decoded is then selected. The 
modifier value is selected from .the provided modifier set based on the 

■ 

intensity representation associated with the image clement and found in the 
representation sequence of the ^encoded imeige block. Once the correct 
intensity modifier value is selected, the intensity of the image element is 

• * 

« • 

modified with this value, 

• • • 

; 

The selection of intensity modifier and modification of the intensity are 

• • • • = 

preferably performed for all image elements that should be decoded in the 
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for all image blocks that comprises image elements that should be decoded. 

Thereafter, a decoded representation of an original image, or a portion 
therjsof, can be generated based on the decoded image elements and blocks. 
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The color representation is preferably generated by expanding the three 4-bit 
color components of the color codeword into three 8-bit components. The 
resulting 24-bit color is then assigned to the image element(s) of the image 
block that are to be decoded. The intensity of the image elements is 
preferably modified by adding or multiplying the intensity modifier to each 
color component, or each color component is othcirwise modulated with the 
intensity modijQer. Thereafter, the resulting intensity modified color 
components are clamped between a minimum and maximum threshold 
value. 

The image encoding and decoding of the invention can be applied to several 
different types of images, including ID, 2D and 3D ""synthetic" images, 
photos, text, games, 3D maps and scenes, 3D messages, e.g. animated 

■ 

raessages, screen savers, man-machine interfaces (MMIs), etc. 

Due to the small (32'-bit) size of an. encoded image block, the invention is well 
adapted for thin clients with limited memory capacity and bandwidth. In 
addition, the encoding is very fast so that it can be performed also in 
terminals with low clock frequencies. Furthexrmore, the decoding can 

* 

» 

extremely simply be implemented in e.g. hardware using only a few standard 
components. 

The invention offers the following advantages: 

Well adapted for matching the human visual system, since the 
luminance component is preserved better than the chrominance 
components; 

Gives high quality (peak sigrial/noise ratio) for different image types; 
Hardware implementation of decoding is extremely simple; 
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Encoding is very fast^ which enables implementations also in tenhinals 
with low dock frequencies; 

Exhaustive encoding is possible at feasible speed on a computer; and 
Encoded image data has a size that is suited for thin clients with 
5 limited memoiy capacity and bandwidth. 

k 

Other advantages offered by the present invention will be appreciated upon 
reading of the below description of the embodiments of the invention. 

10 SHORT DESCRIPTION OF THE DRAWINGS 

The invention together with further objects and advantages thereof, may best 
be understood by making reference to the following description taken 

i 

together with the accompanying drawings, in which: 

' I ' ' 

15 Fig. I is a flow diagram illustrating an image encoding method according to 

the present invention; 

Fig, 2 illustrates an embodiment of an image block according to the present 
invention; 

20 

Fig, 3 illustrates another embodiment of an image block according to the 
present invention; 

Fig. 4 illustrates an embodiment of an encoded representation of an image 
2 5 block according to the present invention; 



Fig. 5 is flow diagram illustrating the color codeword determining step of Fig. 1 
in more detail; 

Fig, 6 is flow diagram illustrating the intensity codeword providing step and 
intensity representation selecting step of Fig. 1 In more detail; 
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Fig. 7 is a flow diagram illusfxating aa image decoding method according to 
the present invention; 

Fig. 8 is a flow diagram illustrating the steps of providing intensity modifier set 
and generating color representation of Fig. 7 in more detail; 

Fig. 9 is a flow diagram iUustrating the intensily modifying step of Fig. 7 in 
more detail; 



10 



Fig. 10 schematically illustrates an example of a user terminal with an image 
encoder and decoder according tp the present invention; 



15 



Fig. 11 is a block diagram schematically illustrating an embodiment of an 
image encoder according to the present invention; 

Fig, 12 is a block diagram schematically illustrating another embodiment of 
an image encoder according to the present invention; 



20 



Fig. 13 is a block diagram schematically illustrating an embodiment of a block 
encoder according to the present invention; 



Fig- 14 is a block diagram schematically illustrating another embodiment of a 
block encoder according to the present invention; 



25 
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Fig, 15 is a block diagram schematically illustrating the color quantizer of the 
block encoder of Figs. 13 and 14 in more detail; 

Fig. 16 is a block diagram schematical^ illustrating an embodiment of an 
image decoder according to the present invention; 

Fig. 17 is a block diagram schematically illustrating another embodiment of 
an image decoder according to the present invention; 
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Pig. 18 is a block diagram schematically illustrating an embodiment of a block 
decoder according to the present invention; 

Fig. 19 is a hardware block diagram schematically illustrating an embodiment 
of a block decoder according to the present invention; 



10 



Fig. 20 is hardware block diagram illustrating an embodiment of the bit 
extenders of Fig. 19 in more detail; 

Pig. 21 is hardware block diagram illustrating an embodiment of the table 
look-up of Fig. 19 in more detail; and 



15 



Pig. 22 is hardware block diagram illustrating an embodiment of the clampers 
of Fig- 19 in more detail- 

DETAILED DESCRIPTION 

Throughout the drawings, the same reference characters will be used for 
corresponding or similar elements. 



20 



25 
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The present invention relates to image' and graphic processing, and in 
particular to encoding or compressing images and decoding or 
decompressing encoded (compressed) images. 

Generally, according to the invention, dxiring im^e encoding, an image is 
decomposed or divided into a number of image blocks. Each such image 
block then comprises multiple image elements having, among others, a 
certain color. The image blocks are then, encoded to generate an encoded 
representation of the image. 

« > 

When an encoded image or graphic primitive subsequently is to be rendered, 
e,g, displayed on a screen, the relevant image elements of the encoded image 
blocks are identified and decoded. These decoded image elements are then 
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used to generate a decoded representatipn of the original image or graphics 
primitive. 
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The present invention is well adapted for usage "with three-dimensional (3D) 

graphics^ such as games, 3D maps and scenes, 3D messages, e.g. animated 
messages, screen savers, man-machirLe interfaces (MMIs), etc., but is not 

■ * • 

limited thereto. Thus, the invention could also be employed for encoding 
other types of images or graphics, e.g. one-dimensional (ID) or two- 
dimensional (2D) images. 

■ . * • ' 

In 3D graphics processing, typically several triangles are created and the 
corresponding screen coordinates of ilie comers of these triangles are 
determined. Onto each triangle," an image (or portion of an image), or a so- 

called texture, is mapped C^glued"). The management of textures is, though, 

•* .■ 

costly for a graphic system, both in terms of utilized memory for storage of 

• 

textxires and in terms of memory bandwidth during memoay accesses, when 
textures are fetched from the memory. This is a problem particularly for thin 
cUents, such as mobile units aiid telephones, with limited memory capacity 

* 

and bandwidth. As a consequence, a texture or image encoding scheme is 
often employed. In such a scheme, a texture is typically decomposed or 
divided into a number of image blocks comprising mxaltiple texels. The image 
blocks are then encoded and stored in .a memory. Note that the sisie of an 
encoded (version of an) image block is simaller than the corresponding size of 
the uncoded version of the image block. • 

• .* 

In the present invention the expression "image element" refers to an element 
in an image block or encoded represeniation of an image block. This image 
block, in turn, corresponds to a por^ibn of an image or texture. Thus, 
according to the invention, an image- element could be a texel (texture 
element) of a (ID, 2D or 3D) texture or a pisel of a (ID, 2D or 3D) image. 
Generally, an image element is characterized by certain image-^element 
properties, such as a color value. Furthermore, in the following, the term 
*'image'* is used to denote any ID, 2D or. 3D image or texture that can be 
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encoded and decoded by means of the present invention, including but not 
limited to photos, game type textures, text, drawings, etc. 



15 



Fig. 1 illustrates a (lossy) method of encoding an image according to the 
present invention. In a first step SI, the image is decomposed or divided into a 
number of image blocks. Each such image block then comprises multiple 
image elements. In a preferred embodiment of the invention, an image block 
comprises eight image elements (pixels or texels) and has a size of 2"^ x 2" 
image elements, where m«3-n and n"0, 1, 2, 3. More preferably, n is 1 or 2. 
Figs. 2 and 3 schematically illustrate two examples of an image block 600 with 
eight image elements 610 according to the present invention. In Fig. 2, the 
height is two image elements 610 and the width is four image elements 610, 
Le. m-1 and n«2, whereas for the image block 600 in Fig. 3 m=2 and n=l- 
Retuming to Fig. 1, the whole image block is preferably decomposed into (non- 
overlapping) image blocks in step SI. However, in some appUcations, only a 
portion of an image is encoded and, thus, only this portion is decomposed into 
image blocks, 
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The following steps S2 to S4 pcrforai an encoding or compression of the image 
blocks. Firstly, in step S2, a color codeword is determined for an image block. 
This color codeword is a representation of the colors of the image elements in 
the image block. In a preferred embodiment, the color codeword is a 
representation of an average color of the image elements of the block. The 
color could be a RGB (Red, Green, Blue) color, a color in the YUV space or 
YCrCb space, or any other proprietary color space used in image and graphics 
processing and management. The color codeword is preferably in the same 
color format (space) as the image. However, in some cases, it may be useful to 
convert the image to a different color format, i.e. having the color codeword in 
a first color space and the original image in a second different color space. The 
color codeword is preferably a 12-bit color-representa.tion sequence. For 
example, a RGB color codeword could comprise 4 bits for the red color 
component, 4 bits for the green component and 4 bits for the blue component. 
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Correspondingly, a YUV color codeword, could include 6 "bits, 3 bite and 3 bits, 
respectively, for the three different components. 

Note that one and the same color codeword (i.e. color representation) is 
generated for an image block, Le, for all the image elements of the image 
block. 



10 



Thereafter, an intensity codeword is provided in step S3. This intensity 
codeword is a representation of a set of multiple intensity modifiers that are 
used (during decoding) for modifying the intensity of the iraage elements in the 
image block- 



15 
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In a preferred embodiment of the invention, the intensity codeword is an 
intensity index allowing identification of an intensity modifier set. This index 
could then identify or point to the set in a table or codebook comprising 
several different intensity modifier sets. Each set comprises two or more 
intensity modifier values, preferably at least four modifier values. In addition, 
the modifier values of a set are prefearably mathematically complementary 
values, i.e. each set is preferably symmetrical. For example, a possible 
intensity modifier set could be [-a, -b, b, a], where a and b are positive integers 
and a>b. 



25 
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The intensity table preferably comprises sets including small intensity modifier 
values, which are adapted for allowir^ representation of smoothly changing 
surfaces. In addition, the table preferably also comprises sets that include 
large intensity modifier values, which are adapted for allowing representation 
of sharp edges. 

The actual intensity modifier values of the sets in the table can be foxmd by 
starting with random values and then optimising these values using a number 
of different optimization schemes and algorithms, such as versions of the 
LBG-algorithm [Linde, Buzo and Gray) [6], simulated armealing and 
coordinate search, which are known to a person skilled in the art. A handful of 
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images of different types e.g. photos, game type textures, text, etc., can be 
used as training data. 

In order to make a hardware implementation of the intensity table less 
expensive, the intensity modifiers of a set can be forced to be symmetrical, as 
was discussed above, and/ or the intensity modifiers of a given set coidd be a 
copy of intensity modifiers of another set modified by a factor, e.g, two. 

Table 1 illustrates a presently preferred example of an intensity table 
comprising 16 sets of intensity modifiers, with four modifier values in each 
set. 



Table 1 



♦ 
♦ 



« 



15 



♦ » 



Set 


Codeword 


. Intensity modifier value 


1 


OOOObin 


-8 


-2 


2 


8 


2 


OOOlbia 




-4 


4 




3 


OOlObin 


-31 


-6 


6 


31 




OOllbia 


-34 • 


-12 


12 


34 


5 


oioobiix 


•SO 


-8 


8 


50 


6 


OlOlbia 


"47 


-19 


19 


47 


7 


OllObiii 


-80 


-28 


28 


80 


8 


Olllbin 


-127 


-42 


42 


127 


9 


lOOObiaL 


-16 


-4 


4 


16 


10 


lOOlbtn 


-24 


-8 


8 


24 


11 


lOlObto 


-62 


-12 


12 


62 


12 


lOllbitt 


-68 


-24 


24 


68 


13 


llOObin 


-100 


-16 


16 


100 


14 


llOlbin 


,94 


-38 


38 


94 


15 


lllObin 


-160 


-56 


56 


160 


16 


llllbia 


-254 

* 


-84 


84 


254 


In Table 1, the intensity modifier sets 9 
a factor of two. 

1 


• 

-16 are a copy of sets 1-8 

4 , 


multiplied by 
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If the intensity table comprises at most 16 different intensily modifier set, the 
intensity codeword is preferably a 4-bit index (OOOOwa- 1 1 1 Itia) identifying one 
of the (16) sets, e.g. [-8, -2, 2, 8] for codeword OOOObin (0000 base 2), of the 
table. Due to careful choice of the modifier values in the sets (symmetrical sets 
and half of the sets are a factor two of the remaining half), the entire Table 1 
can be reconstructed using only. 16 modifier values, and the renaalning 48 
values could be calciilated therefrom. 

t 

a 

The present invention is, though, not limited to usage of Table 1, but could 
use other tables with other intensity modifier sets and values. Furthermore, 
for more or less than 16 sets in a table, the size of the intensily codeword 
might have to be changed. For example, if the table comprises two {3-4, 5-8 or 

■ 

more than 16) intensity modifier, sets, the codeword size could be limited to 
one bit {two bits, three bits or more than four bits). In addition, the number of 
intensity modifier values per set could dif6er from fotnr, e.g. five values could be 
used per set, giving an esiample of {-8, -2, 0, 2, 8]. The intensily values of the 
sets in the table couM be determined using several different types of images as 
training data, as was discussed above. However, if osntly a specific image type is 
to be encoded tiie modifier values could be determined using training data 
corresponding to that image type, i.e. giving an intensity table dedicated for a 
specific ime^e type. It could also be possible to have an intensity table with 
intensity modifier values adapted for a specific image. In these cases, Le. table 
dedicated for image or image type, it mi^t be necessary to include the 
intensity modifier values of the table in the compressed file of encoded image 
blocks or otherwise associate tJiem therewith. 

In addition, the intensily codeword does , not have to be an index or pointer to 
an intensity modifier set in a table, but could actually be an intensity modifier 
set itself, e.g. comprises two modifier values, such as 2 and 8, and where the 
other modifier values, such as -2 and can be determined from these two 
values- 



Note that one and the same intensily modifier set is used for the im^e block. 



•03 12/19 14; 06 FAX +48 18 153080 ARCS PrtCENT AB -» PATENTVERKET ©OlS 

+46 18 153050 ''■ ink. L Pa'ent- och reg.V2rket 

2003 1 9 

Once, the intensity codeword is provided in step S3, a next step S4 selects 
intensity representations for the image elements in the image block. Each 
such intensity representation is associated with one intensity modifier value 
from the intensity modifier set provided in st^p S3. In other words, the 
5 intensity representation allows identification of which intensity modifier of the 

set to use for a specific image element of the block. 

In the case of an intensity modifier set comprising four modifier values, such 
as -8, -2, 2, 8, the intensity representation could be a 2-bit sequence 
10 identifying one of these four values, e.g, Ubin corresponds to -8, lObin 

corresponds to —2, OOwn corresponds to 2 and 0 Ibai corresponds to 8. If more 
than four intensity modifier values are used per set, then more than two bits 
sure required for each intensity element to identify correct modifier. 

15 Step S4 is preferably repeated for all image elements in the image block 

(schematically illustrated by line 1), The result of the encoding of steps S2 to 
S4 is an encoded image block or more precisely an encoded (compressed) 
representation of the image block. Such an encoded block representation 700 
is illustrated in Fig, 4. The representation 700 (encoded image block) 

2 0 comprises the color codeword 710, the intensity codeword 720 and a sequence 

or bitmap 730 of intensity representations (preferably one intensity 
representation for each image element in the block). Notes that the mutual 
order of the color codeword 710, intensity codeword 720 and intensity- 
representation sequence 730 of the encoded image block 700 may differ from 

25 what is illustrated in the figure. 



* m 

* 



MO* 



If the image block comprises eight image elements (see e,g. Figs. 2 and 3) and 
each intensity representation is 2 bits, the sisse of the sequence 730 is 16 bits. 
Furthermore, assume that corresponding sizes of the color and intensity 
3Q codewords are 12 and 4 bits, respectively. The total size of the encoded 

representation 700 of the image block is then 32 bits and a compression rate 
of 4 bits per pixel (image element) (bpp) is obtained. This small (32-bit) size of 

■ 

the representation 700 is well adapted for thin clients, such as mobile units, 



I 
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which typically have menioiy busses of ^6 or 32 bits. As a resixlt, only one or 

* I 

at worst two memory accesses are thin needed to read ou.t the encoded 
representation 700- 



Returning to Fi^. 1, steps S2 to S4 are preferably repeated for all image 
blocks provided during the decomposing of step SI (schematically illusirated 
by line 2). The result is then a sequenc^ or file of encoded image blocks. The 



resulting encoded image blocks (encoded representations of the image 



blocks) could be ordered in a ffle from ;left to right and top to bottom m the 
same order in which they were broken down in the block decomposing of 

Step SI. The method then ends. 

The encoded image could then be provided to a memory for storage therem 
until a subsequent rendering, e.g. disiplay, of the image. Furthermore, the 
encoded im^e could be provided, as a signal of encoded block 
representations to a transmitter for ; (wireless or wired) transmission to 
another imit. 

Fig. 5 illustrates an embodiment of stip S2 of Fig. I in more detail. In step 
SIO, an average color of the image elements in the image block is 
determined. In the following, it is assdcned that the color of a pixel or t«sei 
(image element) of an image is represeiited by 24 bits of RGB color, i.e. 8 bits 
of the red component, 8 bits or the gr«en component and 8 bite of the blue 
component. However, the invention is jiot limited to this particular example, 
but can be api^cable to any color representation of pixels and texels. The 
average color { R , G , B ) is then determined as: 



R 



ii 

_lj 



N 




U) 
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where Ri, Gi, Bi arc the R, G, B component of image element i and N is the 
total number of image elements in the image block. 



10 



15 



20 



Once the average color (R,G,B) is determined in step SIO, a next step Sll 
quantizes the average color. The (24-bit) average color is preferably quantised 
into a 12-bit sequence (color codeword). In other words, each 8-bit average 
component is quantized into a 4-ibxt average coraponenL For example, if the 
average color R,G ,B is calciolated to: 



"178" 




B2 




"10110010" 


88 




58 

■ 




01011000 


21 




IS- 


• 

hex . 


00010101 



bin 



^ rs • 

a 4-bit quantized version (RjiGjBj could be generated from: 



"170" 




"aa" 


* 


"loioioio' 


85 




55; 


• • 


OlOlOIOl 


17 






• 

hex 


00010001 



i.e. [A, 5, l]iiex«[1010, 0101, Op01]bto could be used as a (12-bit) color 
codeword. The method then contiiiues to step S3 of Fig. 1 - 

Pig. 6 illustrates an embodiment :of steps S3 and S4 of Fig. 1 in more detail. 
Step S20 investigates the different intensity modifier sets of the table and the 

* 

different modifier values of the s;ets and calculates an error value for each 
such modifier set and modifier Value test. Based on these error values, a 
modifier set and intensity modifidr [values of the set that results in a smallest 
error value is selected in step S21. This is described in more detail below. 
The method then ends. 



2& 



» • 



• 



■ 

In the following, encoding of image blocks is further discussed in more detail 

» •* 

in connection with three different examples. In these examples, an intensity 
table corresponding to Table 1 is io^e 



5 
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Simple encoding 

In order to encode an image block according to this embodiment of the 
invention, basically a color codeword and a correct intensity modifier set are 
selected. Once this is done, encoding of each image element in the image 
block is done by trying all the four intensity modifiers of the set and 
calculating the error. Assimie that the original (24-bit) color of an image 
element is (R, G, B) and that the color codeword (quantized average color, 12 

bits) is (R,G,B) and that the chosen modifier set is [~a, -b, b, a]. An error 
value could then be determined as: . 



(2) 



15 



20 



where a e [-a, -b, b, a]. For each of the image elements in the image block, 
the a that ininimizes the error 8^ is selected- This could be implemented by 
calculating the error value for a first selection of modifier values for the 
image elements and store this first error value (and the chosen selection of 
modifier values). Thereafter, an error value is calculated for a different 
selection of modifier values. This error value is then compared to the stored 
value. If it-is smaller than the stored value, the stored value is replaced by 
this new error value and the modifier values used in the calculation of this 
error value are also stored- This is then repeated for all combinationB of 
modifiers and sets. 



25 



Equation (2) will give the best performance in terms of Peak Signal/ Noise 
Ratio (PSNR), since it will mininaize.the mean square error in the image. 
Perceptually, however, it roay not produce the best result. For some pixels 
(unage elements) px and p2, where pi. seems brighter than p2 in the original 
manage, p2 may seem brighter than pi in a decoded version of the image. The 
reason for this is that the red, green and blue components do not give equal 
contributions to what the himxan visual system perceives as intensity. Since 
the green component gives a disproportionate contribution to the perception 

< 

of intensity, its approximation should be represented in higher precision 



* ■ • 
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(more acciorate) than those of red and blue. As a result, a -^tigHWW ^SsP 
value could be employed, such as: . * 



* 



6^ 



* * 



(3) 



■ I 

• 

• 4 



where wr, wo, wb are different weights for the color components. In addition, 

5 9 

WG is preferably larger than wr and wb. For example, Wj^ = — , = — and 

16 16 

2 

Wb = — , or WR'^0.299, wg^O.587 and wb=-0. 1 14. 
16 



10 In this simple encoding, an average color of eight image elements in the 

block, quantized to 4 bits per color component, is used as color codeword. 

■ 

The correct intensity modifier set is then chosen by exhaustive search, i.e. all 

the 16 sets in the table are tried, and the set that minimizes the error value 

is selected. This requires 16x4=64 devaluations per image element. If weights 

5 9 2 • 

15 are w,^ — , w^ = — and w^ = — ; , integer arithmetics can be used and the 

16 16 16. 

* 

■ • 

encoding becomes fast- For this selection of weights, encoding of an image of 

a 

128x128 pixels (image elements) uisir^ the simple encoding takes around 60 
ms on a 1 .2 GHz PC laptop. . . 

2 0 Exhaustive encoding 

« - • 

In the simple encoding described . above, the quantized average color was 

> 

simply used as a representation (color codeword) of the colors of the image 
- . : elements in the image block. In this embodiment of the exhaustive encoding 

: according to the invention, both the: colors and the intensity modifier sets 

e 

:2J5 (including the modifier values) are chosen, i-e. eveiy possible combination is 

* ^ '. 

tried. For a given image element, a further iteration through all 12 bits of 
color is added in addition to the previous iteration of all 4 bits of intensity 
modifier set and the 2 bits of intensity representation, which together gives 
2^® steps. Encoding an image of .12S x 128 pixels takes approximately 5 
30-. minutes using the same PC laptop as for the simple compression- Although 
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this migjit be fer too long for run-time appUcalions, it is not prohibitive for 
off-line encoding. Comparing the results from the exhaustive encoding with 
the ones from the simple encoding above shows a difference in PSNR of 
approximately 1.5 dB. Visually the' images differ in that some areas in the 
images encoded with the simple encoding scheme show color shifts. Single 
image blocks in otherwise gray areas suddenly get a slight green shade. The 
reason for this that in the simple scheme the R, G and B components of the 
color codeword are quantized individually, which is suboptimal. 

Combined quantization 

As for the simple encoding scheme, ithis embodiment of the present invention 
starts with a (24-bit) average color •(i:,U,B)» color components of 

this average color are quantized together with the intensity components, i.e. 

selection of intensity modifier sets and values. 

. • * * • 

■ 

If Riow and Fhigb denote the 4-bit qiuantization levels or values that are 
directly below and above R, respectively so that R^., ^ R R^^ . The task is 

then to choose R as either Fi«w or Rhi^. The same is true for the green and 

blue components. 

Firstly, the error value is calculated with (R,G,B)=(Riow, Glow, Biow) 



This can be simplified into; 



where 5j,=Rt^-R, 8q=G,„«-G a 
(the intensity modifier) can be 



+ a-G)'+(B,^+a~B)= 



(4) 



= (5r + a)^ + (&s + a)- + (5^ + a)' , 



(5) 



and 6b = B,„„-B. Fiaxther assume that a 



Innr 



chosien freely, Le. is equal to the optimal 
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85 : 



5„ +S«» 
a = — 5 ^ i.. Inserting this 



simplification: 



optimal a into equation (5) gives after 



20Go I 9 



(6) 



where § is the expression in the brkckets. 

* • * ■ 

•* * . 

'• '.' • 

However, if the higher value is ixistead chosen for the red component, i.e. 

• ■ 

(R,G,B)=(Rijigh, Glow, Blow), and ithe fact that R„^-R=17+5ft is used, 
equation (5) can be rewritten as: 



«'=((SR+17) + a)f + 



This expression can further be 



(5< 



simplified, by inserting the optimal 



a = 



'n 



for this case, into: 



-(§ + 17[17 + 28a-5o-8j). 



+ a)^ + (Sb + a)' 



(7) 



(8) 



In order to determine which of these Itwo quantized colors (color codewords) 
(Rlow, Glow, Blow) or (Rhigh, Glow, Blow) i^: the best, i.e. gives the smallest error 
value, the extra expression in the ;.straight brackets of eqxiation (S) is 
investigated. In other words, if 17-»^24-Sc-5b <0, (Rhigh, Glow, Baow) shoiild 
be chosen, else (Ri«w^, Qtow, Blow) is jchjpsen (in the case 17+28^ -Sq -8^ =0, 
either codeword coiild be chosen). i This procedure is then repeated for all 
possible combinations of low and^ high quantisations for the three color 
components- The result is presented in- Table 2 below. 
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TablSe 2 



Color codeword 



RloWa OlOWr Blow 



Rlii^, Glow, Blow 



Rlow, Ghi^a Blow 



Rlow, Grlow^ Bhigl^ 



Rlowy Ghieli, Bhigfct 



Riii^, Glow, BbigH 



Rhigb, GhigK, Blow 



Rm^, Ghigb., BhisU 



Error value 



3^ 



(§+17[17+25j,-8s-5b]) 



- (£ + 1 7[1 7 + 2So - 5r - S J) 
3 

- + 17[17 + 2Sb - - So ]) 
3 

|(^+17[17-25r + 8^+63]) 
^(4 + 17[17-25o+5r + 5b3) 
i(§ + 17[17-25B + 5a + SoD 



3 



Note that ^ is not required to be expUditly calculated, only the expressions in 
the straight brackets of Table 2 hav^to be calcialated in order to select the 
quantization levels (color codeword) to use. Further note that color codeword 



(Rlow, Glow, Blow) and (Rhs^, Guigh, 
under the assumption that any a 
In reaiity, however, a is limited to 



BMgu) give the same error value. This is 
(intensity modifier value) can be reached. 
: the intensity modifier values of the used 
modifier seUs), e.g. the modifier Values of Table 1. According to Table 1, 
smaller modifier values (a) can be speiCified with greater accuracy than larger 
values, which means that it is betfer |to choose (Rhigh, GfaJst, Buigh) rather than 
(Rlow, Glow, Blow) if (R,G,B) is close^ to (Rhigu, Gbigh, Bhigh) than (Riow, Glow, 
Bw), and vice versa. The combined lijuantization of this embodiment of the 
invention increases the PSNR withj; about 1 dB compared to the simple 
encoding. Thus, the PSNR is only! aiout 0.5 dB lower than the result from 
(opla,, «*au..v. 4>^. T^e total encoding ^ was not 
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measurably changed compared w^tli | the simple encoding, i.e.' an image of 
128 X 128 pixels is still compressed iii about 60 ms- 

' mi 

• • v.: 

It is also possible to build a taiblS corresponding to Table 2 using the 

■ ' ■*'! 

5 perceptually weighted error measuite :|rom equation (3), The optimal a is then 
a for (it,G, B).?=(Riow, Gi«w,, BiH- If the error values for 

. • ••• 

Rlow, Glow, Blow and Ruigh, Ghigh,i \ Bhish are "normalized" to 0 (basically 

I B ^ *• 

• ' ••• 

coiresponas lo suDtxacting from the error value for all different 

valiants of R, G and B in Tabic 2) |he following table of normalized error 
10 values is obtained. Note that the normalized error values for the color 

codewords other than Riow, Giow,;.iBtow and Rhigii, Ghi^, Bhigh of Table 3 
correspond to the expressions in the straight brackets of Table 2. 

••: 

^atiie 3 



• • ■> 



• ■ •4* 



Color codeword Nor!nali2;ed error value 



RloW, GloWj Blow 


• 

• 

• 


;;• 0 






Rhigh, Glow, Blow 






-5k)-2w|(5^ 




Rlow, Gbigh, Blow 








-5o)] 


Rlow, Glow^ Bbigh 






-5b)-2w^(5^ 




Rlow, Ghigh, Bhigh 


* * 


7<H+w|) + 2w^(6o 






Rhigh, Glow, Bhi^ 


■ ■ 




-6«)+2w|(6b 


-Sa)] 


Rtkl^f Ghigh, Blow 






-SB)+2w^(6a 


-5b)J 


RjMgti, Ghigh, Bhigh 


• 

* * 
• 


• 0 







. ! •;.•! 
•.: 

The color and intensity codewords ito use can then be determined bv 
comparing the "normalized" error valines and selecting the codewords that 
result in a smallest error value. : j % 



• • •! 



- - ~ ^ . . 

^RJ Fig. 7 illustrates a flow diagram of ^. ifeiethod of decoding an encoded image 

or encoded version of an original image according to the present invention. 
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The encoded image basically comprises several encoded representations of 
image blocks, such as representations 700 of Fig. 4. These encoded block 
representations are preferably generated by the image encoding method 
discussed above in connection with Fig. 1 . 

5 y 

The method generally starts by identifying encoded image block(s) to decode. 
It coTold be possible that all encoded image blocks of an encoded image 

• • 

should be decoded to generate a decoded representation of the original 
image. Alternatively, only a portion of [the original image is to be accessed. As 

♦ 

10 a consequence, only a selected numtier of image blocks have to be decoded 

(or more precisely, a selected amouiit of image elements of certain, image 

i 

blocks have to be decoded). 

■ 

■ 

• 

. Once the correct encoded (represehta.tion(s) of) im^ block(s) is identified, 
15 step S30 provides an intensity modifier set. This modifier set is provided 

based on the intensity codeword in f. the encoded representation. This set 
provision is prefia-ably performed byiadentifying, hy means of the intensity 
codeword, an intensity modifier set from a table, e.g. Table 1 above, 
comprising mxiltiple modifier sets. However, in some applications it might be 
2 0 possible that the intensity codeword : itself comprises the modifier set and 

that no table look-up is required. - 

■ 

In a next step S31, a color representaition is generated for at least one of the 
im^e elements of the ixixage block (i;e, for the image element(s) that should 
25 be decoded). This color generation is performed based on the color codeword 

* * ' 

: in the encoded block representation:: In step S32, the intensity modifier to 

\ " use for the image element that should be decoded is selected. The modifier 

: v value is selected from the modifier set provided in step S30 based on the 

r v, intensity representation associated with the image element and found in the 

3.0. representation sequence of the encoded block representation. Once the 

correct intensity modifier value is selected in step S32, the intensiiy of the 

■ 

image element is modified or modulated with this value in step S33. 
Intensity modification according to the invention refers to modifying, e.g- 



w • • 



ft • « 



03 12/19 14: 0« FAX +46 18 153050 AROS P/irSW AB PAIENTVERKET ®028 

+46 18 153050. |" 

:: '■ In!<. t tent- oci] rsava-ket 

26 

adding or mioltiplying, all color comjionents of the color ^ep^^^SH-i&l^liri^^^n 
the (possibly weighted) intensity modifier value, 

« * 

.** 

Steps S32 and S33 could be performed for several image elements in the 
5 image block {schematically illustrated by line 3). It is anticipated by the 

invention that in some applications, only a single image element is decoded 
from a specific image block, multiple image elements of a specific image 
block are decoded and/ or all the ittiage elements of a specific block are 
decoded. 

- I. II 

10 

Steps S30 to S33 are then preferably repeated for all image blocks that 

1 • * * 

comprise ixxiage elements that should be decoded (schematically illustrated 
by line 4). This means that the loop of steps S30 to S33 could be performed 
once, but most often several times for .different encoded image blocks and/ or 
15 several times for a specific encoded image block. 

i • 
I 

In the optional step S34, a decoded representation of the original image, or a 
portion thereof > is generated based ! on the decoded image elements and 
blocks. Note that in some applications^ several image elements have to be 
2 0 decoded in order to render a single pixel of the decoded representation- For 

example, during trilinear interpolation, eight neighboring image elements are 
decoded and for bilinear interpolation the corresponding number is four 
image elements, which is well known to the person skilled in the art. The 
method then ends. 

25 

Fig. 8 illustrates an embodiment of steps S30 and S31 of Fig. 7 in more 
detail. In step S40, a correct intensity modifier set is identified and selected 
firom the intensity table by means of the intensity codeword. If the intensity 
modifier set stored in the intensity table comprises a first sub-set of modifier 
\0 values, e,g. [a, b], a second sub-set of intensity modifier values can be 

determined from the values of the first sub-set, e.g. [-a, -b]- In a next step 
S41, the quantized color of the cblor codeword, preferably 12 bits, is 

i 

expanded or extended into, preferably, 24 bits. For the case with a RGB 
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color, each quantised 4-bit color component of the color codeword is then 
expaned into an 8-bit color component. This color expansion may be realized 
by multiplying the quantized 4-bit color components by 17 for an 

^S{5 — ' 1 

implementation with 256 different colors { — = 17). This is the same 

16—1 

* 

thing as replicating the 4-bit pattern to the first (top) and last (lower) 4 bits 
of the expanded 8-bit color word. In other words, a 4-bit color component of 
lOlObia is expanded into 1010 lOiObin, If the color codeword instead 
comprises 15 bits {5 bits for each R, G, B component), the expansion woiild 
result in a 30-bit color. This can be realized by multiplying the (5-bit) color 

1024 — ] 

component by 33 — p«33), or replicating the 5-bit pattern to the top 

and lower 5 bits of the rapanded 10-bit color word. In step S42, the 
expanded color is then assigned to the image elements of the image block, 
which are to be decoded. The method then continues to step S32 of Fig. 7. 



Fig. 9 illustrates an embodiment of step S33 in Fig. 7 in more detail. In step 
S50, the identified and selected intensity modifier value is added to the 
expanded color. Thus, this modifier value is added to all components (all 
three for a RGB color) of the color , for the image element. This could be 
implemented as a simple additiojl of the modifier value to all color 
components- However, in some applications it could be preferred to weight 
the modifier value before adding it to the components. In such a case, 
different weights can be employed for the dilEfercnt color components. In an 
alternative embodiment, another type of modification than a simple addition 
could be employed, e.g. multipUcation, XOR or another modification. In such 
a case, the same modulation is performed to all components of the expanded 

» 

color using one and the same intensity ; modifier value, although this value 
may be different weighted for the .components. In a next step S51, the 
resulting intensity-modified color component values are clamped between a 
minimum color threshold and a maximum color threshold. For example, if 
after adding the (possibly weighted) intensity modifier value to a color 
component, the resulting value is smaller than the minimum threshold, the 

• ^ • 

value is clamped to the value of this" threshold. Correspondingly, if the 
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resulting value is larger than the maximusn threshold, the value of the 
threshold should instead be used for that component. A non-limitmg 
example of a minimum and maximum threshold is O and 255. respectively, 
for the case with 256 different colors. Corresponding thresholds for 1024 
colors covdd be 0 and 1023. The method then continues to step S34 of Fig. 7. 

• « 

Decoding an encoded image block wiJl further be illustrated by an example 
herebelow. In this example, an encoded block representation as illustrated in 
Fig. 4 and an image block as illustrated in Fig. 2 are asstwned. 

The encoded representation of the image block is according to 1010 0101 
0001 0111 11 01 10 00 10 01 00 OObin (a517D890iiex), where the bit 0-3 is 
the red component of the color codeword, bit 4-7 is the green component of 
the color codeword, bit 8-1 1 is the blue cbmponent of the color codeword, bit 
12-15 is the intensity codeword and bit : 16-31 is the sequence of intensity 
representations for the image elements of the block. 

t 

The color codeword is then decoded (expanded) to generate the color 
representation of the image block. Each color component in the color 
codeword is in 4 bits, but is expanded to 8 bits by multiplying by 17 
(17=llhex)» which is the same thing as repUcating the 4-bit pattern to both 
the top and lower 4 bits of the 8-bit word: 



25 



M B 



3n 



« 

A A. M 

f *■ - 



Red: aucxxllhex^aahcx <=> 1010 101 Obin <=> 170 

> ■ 

Green: Shexxllhex-SShet <::> 0101 OlOlbia cs- 85 

Blue: Ihw^ 1 lhest= 1 Ibex 000 1 000 lbb» «4> 17 

r 4 

This expanded color is then assigned to the image elements of the image 
blocks grving: 



(170, 85, 17) 


(170, 85, 17) 


(170, 85, 17) 


(170, 85, 17) 


(170, 85, .17) 


(170, 85, 17) 


: (170, 85, 17) 


(170, 85, 17) 
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THe coixect intensily modifier set to ^se is selected from Table 1 based on 
the intensily codeword. As is seen in Table 1 an intensity codeword of 
Olllbiii corresponds to intensity modifiers 1-127, -42, 42, 127]. 

The sequence of intensity representations enables identification of which of 
these four modifier values to use for 
to: 



the different image element according 









1271 








-42 








42 








. 127 , 


representation 


is 


llbin. 



the first image clement: 



'170" 




'-127' 




43 ' 


85 


+ 


-127 




-42 


_17_ 




L-t27. 




-110 



The resulting components are then clamped between 0 and 255» thus ©ving 
(43, 0, 0). The partly decoded image block is now according to: 



(43, 0, 0) 


(170, 85, 17) 


(170, 85, 17) 


(170, 85, 17) 


(170, 85, 17) 


(170» 85, 17) 

• 


(170, 85, 17) 


(170, 85, 17) 



For the next image element, the intensily representation is Olwn, i.e. the 
intensity modifier 127 should be added to all three color components. The 
result after, clamping is (255, 212, :i44). Repeating this procedure for all 
image elements in the block wotjldj create the final decoded image block 
shown below: 



(43, 0, 0) 


(255, 212, 144) : 


(128, 43, 0) 


(212, 127, 59) 


(128, 43, 0) 


(255, 212, 144) . 


(212, 127, 59) 


1 (212, 127, 59) 
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The image encoding (image block eincoding) and image decoding (image block 
decoding or processing) scheme according to tlie present invention coxald be 
provided in a general data processing system, e.g. in a user terminal or other 
unit configured for processing arid/or rendering images. Such a terminal 
could be a computer. However, the invention is well adapted for thin clients, 
such as Personal Digital Assistance (PDA), mobile units and telephones. 
Such terminals are typically characterized by limited memory capacity and 
memory bandwidth, and are powjercd by batteries, i,e, also limited power 
supply- Since both encoding and d^ecodiug according to the present invention 
can be implemented vciy simply ikx hardware, software or a combination of 
hardware and software and an encoded image block preferably only has a 
maximum size of 32 bits, the invention could with advantage be applied to a 
thin client- 



15 



20 



25 

a • • 

# 1T- 



• * V 



IT • 



Fig. 10 illustrates a user termijnal 100 represented by a mobile unit. 
However, the invention is not ; limited to mobile units by coiild be 
implemented in other terminals and data processing units. Oiily means and 
elements in the mobile unit 100 directly involved in the present invention are 
illustrated in the figure, : 

■ 
I 

The mobile unit 100 comprises a (central) processing unit (CPU) 200 for 
processing data, including image data, within the mobile xmit 100. A graphic 
system 130 is provided in the diobile unit 100 for managing image and 
graphic data. In particular, the graphic system 130 is adapted for rendering 

■ 

or displaying images on a connected screen 120 or other display unit. The 
mobile unit 100 also comprises a storage or memory 140 for storing data 
therein. In this memory 140 iraag^ data may be stored, in particular encoded 
image data (encoded image block^) according to the present invention. Due 
to the small total sisse of image blcjcks (32 bits) and high compression rate (4 
bpp), image data can efficiently bje stored in the memory 140 also in cases 
with a mobile unit 100 with limiteci memory capacity. 
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An image encoder 210 according to the present invention is provided in the 
mobile unit 100. This encoder 210 is configured for encoding an image or 
texture into an encoded representation of the image (or texture)- As was 
discussed above, such an encoded representation comprises a sequence or 
file of multiple decoded image blocks. This image encoder 210 may be 
provided as software rimning on the CPU 200, as is illustrated in the figure. 
Alternatively, or in addition, the encoder 210 could be arranged in the 
graphic system 130 or elsewhere in the mobile unit 100. 



10 



15 



20 



An encoded representation of an image from the block encoder 210 may be 
provided to the memory 140 over a (memoiy) bus 150, for storage therein 
until a subsequent rendering of the image. Alternatively, or in addition, the 
encoded image data may be forwarded to an input and output (I/O) xmit 110 
for (wireless or wired) transmission to other external terminals or units. This 
I/O unit 110 can also be adapted for receiving image data from an external 
unit. This image data could be an image that should be encoded by the 
image encoder 210 or encoded ithage data that should be decoded. It could 
also be possible to store the encoded image representation in a dedicated 
texture memory provided, for example, in the graphic system 130, 
Furthermore, portions of the encoded inaage could also, or alternatively, be 
(temporarily) stored in a texture cache mcmoiy, e.g. in the graphic system 
130. 



25 



r » 



t * 



« <k > 
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If the (memory) bus 150 has a maximal bandwidth of 32 bits, a single 
memory access is required to fetch or read out an encoded image 
representation of the invention from the memoiy 140. If however, the bus 
150 has larger bandwidth capacity, e.g. 64 bits or even 128 bits, multiple 
encoded image representations could be fetched in a single memory access. 
For example assume a 64-bit bus 1 50 and image block size according to Fig- 
2. If the image blocks are piled *'on top of each other", an image block 
together with the subsequent image block in the memory 140 wiU form a 4x4 
square of image elements. However, if the blocks are positioned "side by 
side'% the image block together with .the following block will form a 2x8 box. 
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A 4x4 sqtoare is more preferredj since the probability of finding a desired 
image element in the 4x4 square is larger than for the 2x8 box, if some form 
of texture caching system is employed, which is well known to the person 
skilled in the art. 



10 



15 



An image decoder 220 according to the present invention is provided in the 
mobile unit 100 for decoding an encoded image in order to generate a 
decoded image representation. This decoded representation could 
correspond to the whole original image or a portion thereof. The image 
decoder 220 provides decoded image data to the graphic systera 130, which 
in turn tjTpically processes the data before it is rendered or presented on the 
screen 120. The image decoder 220 can be arranged in the graphic system 
130^ as is illustrated in the figure. Alternatively, or in addition, the decoder 

i 

200 can be provided as software runniiig on the CPU 200 or elsewhere in the 
mobile unit 100, 



20 



25 
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The mobile imit 100 could be equipped with both an image encoder 210 and 

* 

an image decoder 220, as is illustrated in the figure. However, for some 

■ 

terminals 100 it could be possible to orily include an image encoder 210. In 
such a case, encoded image data could be transmitted to another terminal 
that performs the decoding and, possibly, rendering of the image. 
Correspondingly, a terminal 100 could only include an image decoder 220, 
i.e- no encoder. Such a terminal 100 then receives a signal comprising 
encoded image data from another terminal and decodes it to generate a 
decoded image representation. Thus, the encoded image signal could be 

r 

wirelessly be transmitted between terminals using radio transmitter and 
receiver. Alternatively, other techniques for distributing images and encoded 
image representations between terminals according to the invention could be 
employed, such as IR-techniques using IR ports and wired transferring of 
image data between terminals. Also memory cards or chips that can be 
connected and exchanged between terminals could be used for this image 
data inter-terminal distribution. 



m 

> » * 
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The umts 110, 130, 200, 210 andj 220 of the mobHc unit 100 may be 
provided as software, hardware or a cbmbination thereof. 

:i 

.• 

i:. 

Fig. 1 1 illustrates a block diagram of an embodiment of an image encoder 

h 

5 210 according to the present inveiitiori. The encoder 210 typically comprises 

an image decomposer 215 for decoiriposing or dividing an input image into 
several image blocks. The decoiiipbser 215 is preferably configured for 
decomposing the image into image blocks comprising eight image elements 
(pixels or texels), i,e. having a gehe:^al size of 8x1 or 8x1 image elements, 

10 more preferably 4x2 or 2x4 imaged, elements- This decomposer 215 could be 

• •. 

adapted for decomposing different iinput images into image blocks with 

different Bizes. For example, for a first image type an image block size of 4x2 

■ 

* * a 

image elements is used, whereas fprlk second type a block size of 8x1 could 
be used. In such a case, the d^cdmposer 215 preferably receives input 
15 information, enabling identification oif which image block format to use for a 

given image, ' 



» > 



This embodiment of the image encoder 210 comprises a single block encoder 
300, This block encoder 300 encode?! the image block{s) received from the 

* • * ** 

2 0 image decomposer to generate encoded block representation{s) . Such an 

image block representation cotiipr|ses a color codeword, an intensity 
codeword and a sequence of intensity^ representations. The overall size of the 
block representation is much smjaUbr than the corresponding size of the 
uncoded image block. The block leniboder 300 is preferably configured for 

2 5 processing (encoding) each imagdi- block from the decomposer 215 

■ • • 

. . sequentially. : . li^ 

« - :. 
* * - . ,,. 

, The block encoder 300 preferably jcomprises, or has access to, an intensity 

table 500 comprising multiple inten&ty modifier sets. The modifier sets of 
?a the table 500 are used during the encoding for the generation of the 

*t intensity, and possibly color, codeword. The intensity table 500 could be 

arranged in the block encoder 300 pr lelsewhere in the image encoder 210. 



« • « 
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The image encoder 210 coiald cpn|prise a single mtensity labt^ ^OCT. 
Alternatively, several different tabled cpuld be arra^iged in the encoder 210, 
where the intensity modifiers of the fables are adapted for different image 
types or a table could be adapted ibr ja specific image. For example, a first 
intensity table could be used durjinl encoding of a first image type, e.g. 
photo, whereas a second table is |ised for encoding an image of a second 
type, e.g. text. However, in order skv^ memory, a single intensity tabte 500 
generated with training data from ^eVpral different image types is preferably 
employed in the encoder 210. f 



• f. 



The units 215 and 300 of the"iin%e encoder 210 may be provided as 
software, hardware or a combinatiot^ thereof. The units 215. 300 and 500 
may be implemented together in;ithl image encoder 210. Alternatively, a 
distributed implementation is also: pc|ssible with some of the units provided 

15 elsewhere in the mobile xmit. E 

.: i: 
I 

* ' ' J. 

Fig. 12 illustrates a block diagram I of another embodiment of the image 
encoder 210 according to the p^seixt invention. This image encoder 210 
comprises an image decomposer Sjislas the embodiment of Fig. 11. which is 
2 0 not further discussed. However, the ^acoder 2 10 includes multiple (M. where 

M is a positive integer larger faianion|) block encoders 300-1 to 300-M. Each 
such block encoder 300-1 to SOO-k basically corresponds to the block 
encoder of the image encoder in jpig. 11. By providing multiple block 
encoders 300-1 to 300-M in tiic Mge encoder 210, multiple image blocks 
25 from the decomposer 215 may be jprocessed (encoded) in parallel, which 

reduces the total image encoding tinip, 

'.* J, 

i: 

Each block encoder 300-1 to SOq-Mf could comprise an intensity table 500. 
The intensity tables 500 in the d^SS^^ encoders 300-1 to 300-M could all 
include identical intensity mo4fie| values. Alternatively, different block 
encoders could include different lables. In such a case, one or several block 
encoders could be adapted for 4 ^^^^ whereas other block 

encoders are adapted for oliie^ image type(s). In an alternative 



•i • ^ 
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implementation, a sin^e intensity ; table 500 is arranged in the image 
encoder 210 and connected to all block encoders 300-1 to 300-M; 



10 



15 



20 



25 



The units 215 and 300-1 to 300-M of the image encoder 210 may be 

• * 

provided as software, hardware or a combination thereof. The units 215, 
300-1 to 300-M and 500 may be imjpilementcd together in the image encoder 
210. Alternatively, a distributed implementation is also possible with some of 
the \mits provided elsewhere in the mobile unit. 

■ 

Fig. 13 illustrates a block diagram of an embodiment of a block encoder 300 
according to the present invention, such as the block encoder of the image 
encoder in Pig- 11 or one of the block encoders in the image encoder of Fig, 
12, The encoder 300 comprises a color quantizer 310 that determines a color 
representation of the colors of the image elements in the image block and 
quantizes this color representation. The color representation is preferably a 
24-bit average color of the image element and is subsequently quantized to a 
12-bit color representation, i.e. the color codeword, by the quantiser 310. 

■ 

■ 

An intensity quantiser 320 is provided in the block encoder 300 for 
identifying an intensity modifier set: to use for a current image block. The 
quantizer 320 is preferably conj&gured for selecting this modifier set from an 
associated intensiiy table 500. The quantizer > 320 then generates an 
intensity codeword that is associated with the selected modifier set. The 

* 

encoder 300 further includes an intensity selector 330 that selects, for the 
image elements in the image block, an intensity modifier from the identified 
intensity modifier set. The color quantizer 310, intensity quantizer 320 and 
intensity selector 330 are preferably ponfigured for combined quantization of 
the color and intensity codeword, as was discussed in detail in the foregoing. 



3.0 
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The units 310, 320 and 330 of the block encoder 300 may be provided as 

^ • • 

software, hardware or a combixiaiion thereof. The tinits 310, 320, 330 and 
500 may be implemented together iii the block encoder 300. Alternatively, a 
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distributed implementation is alsf jpo&sible with some of the units provided 
elsewhere in the image encoder. 

Fig. 14 illustrates a block diagram^ pf ^otiier embodiment of a block encoder 
5 300 according to the present mveqitiori. This block encoder 300 comprises an 

error estimator 340 for estimating; error values for the purpose of selecting 
intensity modifier set and values,;!^, possibly quantized color value, to use 
for an image Wock. This estiniatdr ; 340 is preferably configured for 
calcTolating an error value for a firisl: s^lefction of an intensity modifier set and 
10 values (and color in the case of eihaiistive encoding) for the image elements 

in the image block. This first crroij ya^ue is then stored. The error calculation 
. is then repeated for all possible :pUe<=txons of modifier set and values (and 
color), and after each calculation!. the estimated error value is compared to 
the stored error. If it is smallerj t^an the stored value, it replaces the 
15 previously stored error. In additi^h, the selection of modifier set and values 

(and color) associated with the j e^or value are also stored. Once all 
combinations have been tested, tile Selection resulting in the smallest error 
is used for generation of intehktgf (and color) codeword and intensity 
representations. An intensity mociifier ^t selector 322 and intensity selector 
2 0 330 then selects the modifier set? and: modifier values that are associated 

with the smallest error. The remainiiig -units of the block encoder 300 have 
correspondences iti Fig. 13 arid are not further discussed. 

« • ■ ♦ 

» » • 

The units 310, 320, 322, 330 eiiid >340 of the block encoder 300 may be 
provided as software, hardware "jbr a combination thereof. The units 3X0, 
320, 322, 330, 340 and 500 Jni^y vbe. implemented together in the block 
encoder 300. Alternatively, a distributed implementation is also possible 
with some of the units provided eiseWhere in the image encoder. 



3C0 A preferred implementation of a ^oloi quantizer 3 10 according to the present 

invention is illustrated in the biockUiagram of Fig. 15. The quantizer 310 
comprises means 312 configured ]fo£: determining an average of the colors of 
the image elements in the imagd . bioc;k. This average color is preferably a 
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• * • ' 

RGB color, but coiold be any other -color format used in image processing. 
This determined average color isi then provided to quantizing means 314, 

• * * ■ » 

which quantizes the averse ibolbr. The quantizer 314 is preferably 

* > 

configured for quantizing a 24-bit.vavprgLge RGB color from the color averager 
312 into a 12-'bit RGB color. 



■ • 



The units 312 and 314 of the: bolpr. quantizer 310 may be provided as 
software, hardware or a combination: thereof ► The units 310 and 314 may be 
implemented together in the color quantizer 310. Alternatively, a distributed 
implementation is also possible ^irtthisorne of the units provided elsewhere in 
the block encoder. : l- " 

. I 

* m • * 

'. • 

Fig. 15 illustrates a block diagram of aft embodiment of a block decoder 220 
according to the present invention. ^The block decoder 220 preferably 
comprises a block selector 222; ithait is adapted for selecting, e-g. from a 
memory, which encoded im^e biock(s) that should be provided to a block 

* » ■ 

* « 

decoder 400 for decoding. The block selector 222 preferably receives input 

• • • 

information associated with the encoded image data, e»g. from a header or a 

' • • 

rendering engine. An address of Mn encoded image block having the desired 

• » * » 

image element(s) is then computed* based on the input information. This 
computed address is preferably dependent upon the image-element (pixel or 

• * * 

texel) coordinates within an imagdj. Using the address, the block selector 222 
identifies the encoded image block from the memory. This identified encoded 
image block is then fetched from the storage and provided to the block 
decoder 400. ' 



The (random) access to image elements of an image block advantageously 
enables selective decoding of dnlyi.those portions of an image that are 
needed. Furthermore, the image jtan fee decoded in any order the data is 
required- For example, in texture ;mapping only portions of the texture may 

be required and these portions will generally be required in a non-sequential 

, 

order. Thus, the image decoding of the present invention can with advantage 

■ • • 

by applied to process only a portion or section of an image- 
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• • • • 

The selected encoded image blocfc: ib then forwarded to the block decoder 
400. In addition to the image block, the decoder 400 preferably receives 
information specifying which iiii4ge': elements of the block that should be 
decoded. The information could Specify that the whole image block, Le. all 
image elements therein, shoul^ jbje decoded. However, the received 
information covUd identify only a single or a few of the image elements that 
should be decoded. The block Idecoder 400 then generates a decoded 
representation of the image eleineiit(s) in the block. This decoded 
representation is preferably a P-b|t color, where P is the number of bits per 
image element in the ori^nal image- e.g. a 24-bit RGB color. The block 
decoder 400 preferably comprisesfan intensity table SOO that is used dtinng 

• • • 

the decoding procedure. Altemajtively, this intensity table 500 could be 
provided elsewhere in the image rdecoder 220. Usage of different intensity 
tables for different image types, (iistiussed above in connection to Fig. 11, 

also applies to the image decoder 220. 

• ■ • . 

i ■ < . 

An optional image composer 224 could- be provided in the decoder 220. This, 
composer receives the decoded im^e ;?!leraents from the block decoder 400. 
and composes them to generate aipiad; that can be rendered or displayed on 
a screen. The composer 224 coulid tequire several input image elements to 
generate a, single pixel. This imiige icomposer 224 could alternatively be 
provided in the graphic system. 

• . * 

The units 222, 224 and 400 of the iblbck decoder 220 may be provided as 
software, hardware or a combination ^thereof- The units 222, 224, 400 and 
500 may be implemented togethei; in the block decoder 220- Alternatively, a 
distributed implementation is also possible with some of the units provided 
elsewhere in the mobile unit. 



Fia> 17 illustrates a block diagijanii of another embodiment of an image 
decoder 220 according to the present myention. The block selector 222 and 
image composer 224 are similar |to corresponding units in Pig, 16 and are 
not further discussed* 
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The image decoder 220 comprises i^uitiple block decoders 400-1 to 400-Q (Q 
is a positive integer larger than 6ne).: By having access to multiple block 
decoders 40O-1 to 400-Q, the iikiage: decoder 220 can process (decode) 
mratiple encoded image blocks ii| parallel. These multiple block decoders 
400-1 to 400-Q allow for parallel [processing that increases tlie processing 
performance and efficiency of the image decoder 220. For example, one 
decoded image element is ge4ertdly sufficient for nearest neighbor 
interpolation, whereas four (eight) iimage element are need for bilinear 
(trilinear) interpolation. Each block decioder 400-1 to 400-Q could comprise 
an intensity table 500 used for theid^coding. Alternatively, a single table 500 
is arranged in the image decoder i220>nd connected to all block decoders 
400-1 to 400-Q. The further disc4ssioh of using different types of intenaty 
tables, see above in connection Withl Fig. 12, also appUes for the image 
decoder 220. 

* • 
i . ■• 
i* • • 
! ; *' 

The units 222, 224 and 400-1 to[4qO^Q of the image decoder 220 may be 
provided as software, hardware 6t a combination thereof- The vmits 222, ; 
224, 400-1 to 400-Q and 500 mky ;b<; implemented together in the image 
decoder 220. Alternatively, u distii^utei implementation is also possible with , 
some of the units provided elsewhlre^in; the mobile unit. 

* « 

Fig. 18 is an illustration of an emiodixAent of a block decoder 400 according 
to the present invention. The blobk idi^coder 400 comprises means 410 for 
providing an intensity modifier sLt W an associated intensity table 500 
based on the intensity codeword. Ms .^provider 410 could be configured for 
fetching a first sub-set of modifi*^ values from the intensity table 500 and 
determining a second sub-set of i^odifera based on the first sub-set. A color 
generator 420 generates a single colbr i-epriesentation for all image elements 
in the image block based on the; bolor codeword. This generator 420 

preferably expands the 12^bit color of the codeword into a 24-bit (RGB) color. 

i ' •• 

An intensity modifier selector '^O: is arranged for selecting one of the 
intensity modifier values from th^ r^odifier set provided by the means 410. 
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The modifier selector 430 is configti^d :for selecting correct modifier values 
for the iioage elements in the encoded image block based on the sequence of 
intensity representations. The expanded color from the color generator 420 
and modifier value from modifier selector 430 are forwarded to an intensity 
modulator or modifier 440 tha|t m6difies the intensitjr of the color 
components of the expanded color! with^the modifier value- The modifier 440 
could use a weighted intensity modifier: value, v^dth different weights for the 
different color components. Furthermore, once the color components have 
been intensity modified the modifier 440 preferably clamps the components 
between a maximum and minimurii threshold, e.g. between O and 255. 

The units 410, 420, 430 and 440 jof the block decoder 400 may be provided 
as software, hardware or a combination thereof. The units 410, 420, 430, 
440 and 500 may be implemerited together in the block decoder 400. 
Alternatively, a distributed impleiheritation is also possible with some of the 
units provided elsewhere in the iniage decoder. 

Fig. 19 schematically illustrates k possible hardware implementation of a 
. block decoder 400 according to thle present invention. The input to the block 
decoder 400 is an encoded bloci rtepiresentation 700 comprising a 12-bit. 
color codeword 710 (4 bits for each of the red, green and blue component), a 
4-bit intensity codeword 720 and a 16-^it sequence of intensity 730. 

The color codeword is provided toi, the color generator 420, which is realised 
by three bit extenders 422 to 426. A. first bit extender 422 receives the 4-bit 
red component, a second 424 and^thjrd 426 extender receives the 4-bit green 
and blue component, rcspectivels|. Tht;. output from respective extender 422 
to 426 is an 5-bit color componerft. i^his 8-bit component is simply obtained 
by multiplying the input componfcnt 17, or multiplying the component 
with 16 and then adding the comjpoheiit. Alternatively, the extenders 422 to 
426 could be implemented as bit jshift^rs and OR-gates, e.g. (1011w»«4) OR 

lOllDfa - 1011 OOOObin OR 101l4 =: 1011 lOUbin, where <^4 corresponds to 

shifting a word foxir bits to the left. 
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A modifier selector 430 is implemeJited as a multiplexor 4i^^")^'lf-^iit afiiress 
index is input to this multiplexof 435. Based on tbe address index, the 
multiplexor 435 selects which of 4^ eight image elements to decode. The 2- 
bit intensity representation associated | with the selected image element is 
then forwarded to a table looJc-up f Is! This table look-up corresponds to the 
modifier set provider 410 and inte^isity jtable 500 of Fig. 18. Using the input 
intensiiy codeword and intensity rjepresentation the look-up 415 fetches the 
correct intensiiy modifier value from one of the modifier sets in the table. 
This 9-bit signed (positive or negaiive) kodifier value is then provided to aa 
intensity modifier 440. In this hqu-dware implementation the modifier 440 
comprises three adders 441 to 4;43 and three clampers 444 to 446. The 
modifier value is input to respectivje adder 441 to 443. A first adder 441 adds 
the intensity modifier value to tHe 8-bit red component from bit extender 
422. Correspondingly, adder 442 jand adder 443 adds the modifier value to 
the 8-bit green and blue component from bit extender 424 and 426, . 
respectively. In an alternative impjlemeritation, the adders 441 to 443 can bc; 
replaced with other modifying eleinent^, e.g. multipfiers or XOR gates. The,, 
outputs fi-om the adders 441 to ^43 a^e forwarded to clampers 444 to 446.. 
which clamp the intensity modifiied.cqlpr components between 0 and 255.; 
The output from the clampers 444 to ;446 is the decompressed or decoded- 
24-bit color of the image element, i . 



\ * 

Fis, 20 schematically jfiustrates a- ppsfeible hardware implementation of tbe 
bit extenders 422; 424j 426 of Fik- 19> These extenders receive a 4-bit (red, 
green or blue) color component ahdi .oUtput an extended corresponding S-bit 
color component. In the outpdt S-bit coUir component, the foxir most 
significant bits (MSBs) constitutej the input 4-bit color component, the "fifth 
MSB" corresponds to the MSB jif :the input component, the "sixth MSB' 
corresponds to the "second MSB" of the input component and the remaining 

correspond to the two LSBs of the input 



two least significant bits (LSBs) 
compOTieiit- 
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Fig, 2 1 schematicaUy illustrates a possible* hardware implementation of the 
table look-up 415 of Fig. 19. The three LSBs of the 4-bit input intensity 

codewords are input to two multiplexors 4 jt 1 and 412 for selecting one 7-bit 

' ■ < ■• 

intensity modifier value from eight possible modifier values for each 
multiplexor 411 and 412. From these 16 intensity modifiers the remaining 

• • ■ 

48 values could be calculated if employing an intensity table according to 

Table 1. The selected intensity modifier vialues from the multiplexors 411 

and 412 are input to another mtdtiplexoir 413 that selects one of these 

values based on 1-bit input data (1 bit of liie 2-bit intensity representation) 

from the multiplexor 435 in Fig. The! selected modifier value is then 

■ 

forwarded both to a multiplexor 416 and to 'negation means 414 that negates 
the modifier value. Also this negated: value is forwarded to the multiplexor 
416. This multiplexor 416 selects either the positive 7-bit intensity modifier 
value or the negated value based oh the remaining bit of the intensity 
representation from the multiplexor : 435 'in Fig. 19,. The selected (8-bit) 

• • • * 

modifier value is then brought both to a multiplexor 418 and to a bit-shifter 

417 that shifts the modifier value one bit to the left, resulting in a 9-bit 

■< . ■ 

intensity modifier (corresponds to .a znultipiication of the value, in base ten, 

...... 

by two). The multiplexor 418 then: selects either the S-bit modifier value or ' 
the 9-bit modifier value based on the; MSB ifrom the intensity codeword. The ^ 
result from the selection is the 9-bit. intensity modifier value, out of the 64 
possible modifier values, to use for .:a sjpecific image element. 



25 
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Fig. 22 schematically illustrates a ipossible hardwaxe implementation of the 
clampers 444; 445; 446 of Fig- 19. ?rhe input to the clamper 444; 445; 446 is 

■ • • * 

a 10-bit intensity-modified color component value- The eight LSBs of this 
input value are brought a multiplexor: 447. The other input to the 

multiplexor is the maximum threshold valine (255; 8 bits). The multiplexor 

,'...«• 

447 selects either the 8-bit input lvalue pr the maximum threshold value 

- . . • , m 

based on the second MSB of the inteitsity modified color component. In other 

■ ' * 

words, if this second MSB is equal, tb; one,-: the multiplexor 447 outputs the 
threshold value, otherwise (the second MSB is equal to zero) the 8-bit input 

• * * 

value is output to a second multiplexor 4.48. This second multiplexor 448 
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compares the output from the first multiplexor 447 with the minimum 

♦ ♦ 

threshold value (0; 8 bits) baoed on the MSB of the color component. If this 
MSB or sign bit is equal to one, the output from the first multiplexor 447 is 

* ft 

negative and the minimum threshold valiie should be selected by the second 
multiplexor 448. However^ if the sign bit is zero the output from the first 
multiplexor 447 should also be output from the second multiplexor 448, 



r, 



10 



15 



The hardware solution for. the block encoder 400 in Fig« 19 is veiy simple, it 
basically compiiees only three additions,; one negation and 12 multiplexors if 
the bit extenders 422; 424; 426, tablfe lock-up 415 and clampers 444; 445; 
446 are implemented according to Pig. 20, Fig, 21 and Fig. 22, respectively. 
This should be compared to decompression using the S3TC scheme [3], 
which requires up to 42 additions audi two multiplexors- 

It will be understood by a person skiUed in the art that various modifications 
and changes may be made to the present invention without departure from 
the scope thereof, which is defined by the appended claims. 
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1 - A method of compressing an image block (600) comprising multiple image 
elements (610), said method comprising the steps of: 

determining a color codeword (710) that is a representation of the 
colors of said image elements (610) in said image block (600); 

providing an intensity codeword (720) that is a representation of a 
set of nmltiple intensily modifiers for modii^riutig the intensity of said image 
elements (6 1 0) in said image block (600); and 

selecting, for each image element (610) in said image block (600), an 
intensity representation (730) associated with an intensity modifier from said 
intensily modifier set. 

2. A method of encoding an image, said method comprising the steps of: 

decomposing said image into image blocks (600), each image block 
(600) comprising mialtiple image elements (610); and 

- determining, for at least lone image block (600), an encoded 

representation (700) by: 

- . determining a color codeword (710) that is a representation of 
the colors of said image elements (610) in said image block (600); 

providing an intensity codeword (720) that is a rcpresentationi 
of a set of multiple intensily modifiers for modifying the intensily of said image 
elements (610) in said image block (600); and 

selecting, for each image element (610) in said image block 
(600), an intensily representation (730) associated with an intensity modifier 
from said intensity modifier set. 

• m 
« 

3. The method according to claim 2» wherein said step of deteraiining said 
encoded representation (700) is performed for each image block (600) • 



30 



4. The niethod according to any of the claims 1 to 3, wherein said step of 

determining said color codeword {710)J comprises the steps of: 

, «• 

determining an average color of said image elements (610) in said 
image block (600); and 
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quantizing said aver^ige color.' 



5, The method according to any of the claims 1 to 4, wherein said intensity 
modifier set comprises mathematically complementary intensity modifier 
values. 



10 



6. The method according to any of the claims 1 to 3, wherein said intensity 
codeword (720) providing step comprises selecting said intensity modifier set 
from an intensity table (500) comprising miiltiple intensity modifier sets^ 
whereby said intensity codeword (720) enables identification of said selected 
intensity modifier set from said table (500). 
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7, The method according to claim 6, wherein said steps of selecting said 
intensity modifier set and selecting said intensity representation (730) 
comprise the steps of: 

detemwning^ for each image element (510) ia said image block (600) , 
a difference between an original color value and an average color intensity-; 
modified with an intensity modifier firom an intensity modifier set of said 

« ■ 

intensity table (500); 

estimating an error value based on said determined differences; and 
selecting an intensity modifier set and intensity modifiers that. 

minimize said error value. 



B. The method according to claim 7 
error value, a weight for a given color 
and said average color. 



> wherein said error value is a weighted 
component being equal for said original 



9. The method according to claim &, wherein said steps of determining said 
color codeword (710), selecting said intensity modifier set and selecting said 
intensity representation (730) comprise the steps of: 

detesrmning, for each image, element (610) in said image block (600), 
a difference between an original color value and a quantized color intensity- 
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modified with an intensity modifibr from an intensity modifier set of said 
intensity table (SOO); 

estimating an error value; based on said determined differences; and 

* 

selecting a quantised color, an. intensity modifier set and intensity 
5 modifiers that minimize said error value, whereby said color codeword (710) 

* 

comprises said selected qxiantized 0olor. 

m 
• 

1 • . 

i 

• • 

m * * 

; ■ 

10- The method according to clai:^ 9, wherein said error value is a weighted 
error value, a weight for a given cqlor component being equal for said original 
10 and said quantized color. 

11. The method according to any: of the claims 1 to 10, wherein said image 
block (600) comprises 2°^>:2^ imag^ elements (610), where m==3-n and n=0, 1, 
2, 3. 
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12, The method according to claikn 1, wherein a compressed representation 
(700) of said image block (600)1 is 32-bit sequence, where said color 
codeword (710) comprises 12 bits J said intensity codeword (720) comprises 4 
bits and a sequence of said intensity representations (730) comprises 16 bits, V 

i . 

13, The method according to claim 2, wherein said encoded representation 



(700) of said image block (600) 
codeword (710) comprises 12 bitsj 



is a; .32-bit sequence, where said color 
said intensity codeword (720) comprises 4 



• 

bits and a sequence of said intensity representations (730) comprises 16 bits. 



14. A method of processing a cotnpressed representation (700) of an image 
block (600) comprising multiple! image elements (610), said compressed 

: • • 

representation (700) comprising a bolor codeword (710), an intensity codeword 

* 

N » » ' , I ! 

- (720) and an intensity represeritatiori sequence (730), and said method 

..so oon.pri.hM, th. st^s or: ' \ / : 

providing a set of mijiltiple! intensity modifiers based on said 

i 

intensity codeword (720); j . 

5 

for at least one image element (61Q) in said image block (600); 
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generating a color representation based on said color codeword 

(710); 

selecting an intensity modifier from 5aid intensity modifier set based 
on said intensity representation sequence (730); and 

modifying the intensity: of said at least one unage element (610) 
based on said selected intensity modifier. 



15. A method of decoding an encoded image that comprises encoded 
representations (700) of image blocks (600) comprising mialtiple image 
10 elemeants (6 10),, an encoded representation (700) comprising a color codeword 

(710), an intensity codeword (720) and an intensity representation sequence 
(730), and said method comprisesi for at least one encoded representation 
(700) of an image block (600), the steps of: 

providing a set of multiple intensity modifiers based on said 
15 intensity codeword (720); 

for at least one image element (610). in said image block (600) : 

- generating a color riepresentation based on said color codeword 

(710); 

selecting an intensity modifier from said intensity modifier set 
2 0 based on said intensity representation! sequence (730); and 

modifying the intensity of said at least one image element (610), 
based on said selected intensity modifier; a nril . 

generating a decoded representation of said encoded image by 
composing multiple colored and intensity modified image elements (610). 



16. The method according to claim 14 or 15, wherein said intensity modifier 
set comprises mathematically complementary intensity modifier values. 

17. The method according to any of the claims 14 to 16, wherein said step of 
providing said intensity modifier set comprises selecting, based on said 
intensity codeword (720), said intensity modifier set from an intensity table 
(5O0) comprising multiple intensity inodifier sets. 
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18. The method according to daim 17, wherein said table (500) comprises a 
jBrst sub'-set of intensity modifier sets, said method further comprising 

determining intensity modifiers of intensity modifier sets for a second sub-set 

.» •■ 

based on intensity modifiers of intensity modifier sets in said first sub-set. 
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19. The method according to any of the claims 14 to 18» wherein said 
intensity representation sequence (730) is a sequence comprising, for each 
image element (610) in said representation (700) of said image block (600), an 
intensity index enabling identificatioji of which of said multiple intensity 
modifiers of said intensity modifier setito use for said ima^ element (610). 
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20, The method according to any of the daims 14 to 19, wherein said 
intensity modifying step comprises adding a selected intensity modifier value 
to all color component values of said color representation. 

♦ 

21- The method according to any .of the claims 14 to 19, wherein said 

♦ 

intensity modifying step comprises adding a selected intensity modifier value 
multiplied by component-specific weigjtits to all color component values of said 
color representation* 

* 

22 - The method according to claim- 20 or 2 1, fxirther comprising clamping the 
sums of said intensity modifier value and said color component values 
between a minimum threshold value ^d a maximum threshold value • 

23. The method according to any of the claims 14 to 22, wherein said image 
block (600) comprises 2^x2^ image eljements (610), where m=3-n and n=0, 1, 



3. 
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24, The method according to any :6f the claims 14 to 23, wherein said 
representation (700) of said image block (600) is a 32-bit sequence, where said 
color codeword (710) comprises ! 12 ' bits, said intensity codeword (720) 

' • • • 

comprises 4 bits and said intensity representation sequence (730) comprises 
16 bits, . ; 
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25, A signal representation (700) of aii; image block (600) comprising m^dtiple 
image elements (610), said signal reptetentation (700) comprising: 

a color codeword (710) that is a representation of the colors of said 
image elements (610) in said image block (600); 
5 - an intensity codeword (720): that is a representation of a set of 

maltiple intensity modifiers for modifyiiig the intensity of said image elements 

■' 

(610) in said image block (600); and 

a sequence (730) of intensity representations, where an intensity 
representation is associated> for a imkge element (610) in said image block 
1 0 (600), with an intensity modifier from, said intensity modifier set. 

• <> 

26. The signal representation according to claim 25, wherein said image 
block (600) comprises 2^x2*^ image elements (610), where m=3-n and n=0, 1> 

2 a • • : • 



15 



20 



27, The signal representation according to claim 25 to 26, wherein said signal 
representation (700) is a 32-bit se<3uence> where said color codeword (710) 
comprises 12 bits, said intensity codeword (720) comprises 4 bits and said 
intensity representation sequence {73d) comprises 16 bits, 



28. A system (300) for compressing ah inaus^e block (600) comprising multiple 
image elements (610), said system (300) comprising: 

a color quantizer (3 10) for ;d0termining a color codeword (710) that is 

■ • 

a representation of the colors of said image elements (610) in said iniage block 

... . 

25 (600); 

;/J - an intensity quantizer (320) for providing an intensity codeword 

(720) that is a representation of a;; set. of multiple intensity modifiers for 
I " niodifying the intensity of said image eleioients (610) in said image block (600); 

A ^ tp- , ^ *« ^ 

%" " and ' :: * 

» . •• • 

.ao" - an intensity representatioii selector (330) for selecting, for each 

image element (610) in said image block (600) > an intensity representation 
(730) associated with an intensity modifier from said intensity modifier set. 



m m 



» # 
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^6r decomposing an image into image 
comprising mxxltiple image elements 



an image decomposer (215) 
blocks (600), each image block (600| 
(610); and 

a block encoder (300) for ciidbding an image block (600) to generate 
an encoded representation (700) of saicLimage block (600), said block encoder 
(300) comprising: 

a color quantlaier (310) 
that is a representation of the colors 
image block (600); 

an intensity quantizer (2|20) for providing an intensity codeword 
(720) that is a representation of a uet of multiple intensity modifiers for 
modifying the intensity of said image e|ements (610) in said image block (600); 
and 



|br determining a color codeword (710) 
of said image elements (610) in said 



an intensity representation selector (330) for selecting, for each 
image element (610) in said image block (600), an intensity representation 
(730) associated with an intensity mod fier from said intensity modifier set. 

30. The system according to claim ^8 or 29, wherein said color quantizer 

(310) comprises: 

means (312) for determining 
in said image block; and 

means (314) for quantizing sfiud- average color. 



an average color of said image elements; 



31. The system according to any" cf the claims 28 to 30, wherein said 
intensity modifier set comprises mathematically complementary intensity 
modifier values* 

< 

32. The system accordmg to claim ; 28 .or 29, further comprising an intensity 

table (500) comprising multiple inteiasity modifier sets, and said intensity 

• • • 

quantizer (320) comprises a selector (:B2i2) for selecting said intensity modifier 

» • • 

set from said intensity table (500);, whereby said intensity codeword (720) 

i 
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enables identification of said selected! intensity nxodifier set firom said table 
(500). 

33* The system according to daira 32 ,| further comprising: 



means (340) for dete 



for each image element (610) in said 
image block (600), a difference between an original color value and an average 
color intensity-modified with an intensity modifier from an intensity modifier 
set of said intensity table (500); anid 



an error estimator (340) :ifor 



Estimating an error value based on said 



determined differences, and said Modifier set selector (322) and said 

• * il 

representation selector (330) are cpniigured for selecting an intensity modifier 

set and intensity modifiers that mibimize said error value. 

• 

• ' :• 

• « * * 

34, The system according to claim 33, wherein said error estimator (340) is 

.: - 1; . 

adapted for estimating a weighted'- e^ror value, a weight for a given color 
component being equal for said origms J and said average color. 



35. The system according to claini 32 



Ifurther comprising: 



means (340) for determininfej for each image element (610) in said 
image block (600), a difference betweei^ian original color value and a quantised 
color intensity-modified with an intensity modifier from an intensity modifier 

■ *. 

* :• 

set of said intensity table (500); and ^ jl ' 

■ ■■ ii ■ 

an eiror estimator (340) for ^'stimating an error value based on said 
determined differences, and said icokpf quantizer (310), said modifier set 
selector (322) and said represehtatipn selector (330) are configured for 

. * . ' 

selecting a quantized color, an intensity, modifier set and intensity modifiers 

that mininwbse said error value, where by said color codeword (710) comprises 

• • * 

said selected quantized color, 

« ■ 
* 

♦ 

• - < 

36- The system according to claim- 3 i wherein said error estimator (340) is 

; • J' • , 

adapted for estimating a weighted ; exror- value, a weight for a given color 

• It 

component being equal for said originaij and said quantised color. 



05 



.12/19 14;16^AX..+46 18 153050 ARCS .PM 

H6 18 153050 



0 



,5 



0 



0 



5S 



37. The system according to any of 
block (600) comprises 2*^x2^ image 
2, 3- 



thje dledms 28 to 36, wherein said image 
eleipaents (610), where m-3-n and n=0, 1, 



38. The system according to claim 28 
(700) of said image block (500) is 
codeword (710) comprises 12 bits, sale, 
bits and a sequence (730) of said intens i 



39: The system according to claim 2? 
(700) of said image block (600) is 
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wherein a compressed representation 
a ; $2-bit sequence, where said color 
tiitensity codeword (720) comprises 4 
it^rl representations comprises 16 bits. 



•J 



, x?^rhercin said encoded representation 

• X 

aj;32-bit sequence, where said color 



codeword (710) comprises 12 bits, saic^ intensity codeword (720) comprises 4 
bits and a sequence (730) of said intensity] representations comprises 16 bits. 



40. A system (400) for processing a. 
image block (600) comprising multiple 



coimpressed representation (700) of an 
image elements (610), said compressed 



representation (700) comprising a coior cddeword (710), an intensity codeword 
(720) and an intensity representation sequence (730), and said system (400) 
comprising: :-| 

• • • 

means (410) for providing a.4^t'|of multiple intensity modifiers based 
on said intensity codeword (720); 



a color generator (420) for getierating a color representation for at 
least one image element (610) in said image block (600) based on said color 
codeword (710); I 

a selector (430) for selectinkrfor said at least one image element 
(610), an intensity modifier from said ideiitified intensity modifier set based on 
said intensity representation sequence 



: .'! 



(730); and 

an intensity modifier (440) fcr kftodifying the intensity of said at least 



one image element (610) based on said 



41, A system (220) for decoding an 
representations (700) of image 
elements (610), an encoded 



representfitidn 



sheeted intensity modifier. 



^ri<5oded image that comprises encoded 
bibck^ (600) comprising multiple image 
(700) comprising a color codeword 
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(710), aa intensity codeword (720) arid: an mtensity representation sequence 
(730), said system (220) comprising: ; 

means (410) for providing, fc r &ii image block (500), a set of mialtiple 
intensity modifiers based on said intfer siify codeword (720); 

a color generator (420) for generating a color representation for at 
least one image element (610) in said iiAage block (600) based on said color 
codeword (710); 

a selector (430) for selecting, : for said at least one image element 
(610), an intensity modifier from said identified intensity modifier set based on 
said intensity representation sequence (730); 

an intensity modifier (440) for modifying the intensity of said at least 
one image element (610) based on said selected intensity modifier; and 

means (22S) for composing multiple colored and intensity modified 
image elements (610) to generate a, decoded representation of said encoded 
image, 

42. The system accordiag to claim 4G 6t 41, further comprising an intensity 
table (500) comprising multiple intensity modifier sets, and said means for 
providing said intensity modifier set ii eonfiguured for selecting said intensity 
modifier set from said intensity table 
(720). 



sop) based on said intensity codeword 



43. The system according to any ! 6 f :the claims 40 to 42, wherein said 
intensity modifier set comprises mathematically complementary intensity 
modifier values, 

• * • 

44. The system according to any o: ithe claims 40 to 43, wherein said 
intensity representation sequence (730)1 is ia sequence comprising, for each 
image element (610) in said represents doh (700) of said image block (600), an 
intensity index enabling identificatioh ;6:f which of said multiple intensity 
modifiers of said intensity modifier set to juse for said image element (610). 
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45. The system according to any c if the claims 40 to 44, wherein said 

for adding a selected intensity modifier 



>nfigured 



value to all color component values of said color representation. 



46. The system according to any < 
intensity modiBer (440) is configured 
value raultiplied by component-spedfip 
of isaid color representation. 



the claims 40 to 44, wherein said 
for: adding a selected intensity modifier 
weights to all color component values 



47. The system according to. claim 4J3 or 46, wherein said intensity modifier 
(440) is configured for clamping the s\mas of said intensity modifier value and 
said color representation values between a minimum threshold value and a 
ma?dmum threshold value. 

48. -me ^ acco^, ^ ^ofi..^,0 to 47. .hcreta imag. 
bJock (600) comprises 2°hc2^ image cManents (610), where m-3-n and n=0, 1, 
2,3. 

49. The ^stem according to any of ithe claims 40 to 48, wherein said 
representation (700) of said image block (600) is a 32-bit sequence, where said 
color codeword (710) comprises 12 tdts, said intensity codevrord (720) 
comprises 4 bits and said intensity r^esfentation sequence (730) comprises 
16 bits. 
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ABSTRACT 

In an. kaage-encoding scheme, ad in ^ut iiii.agc is decomposed into several 
image blocks (600) comprising multiple^ image elements (510), e.g. pixels or 
texels. The image blocks (600) ard :then encoded into encoded block 

representations (700J. Such a block representation (700) comprises a color 

' .• • . * * 

codeword (710), an intensity codeword (720) and a sequence (730) of intensity 
representations. The color codeword (719) representation of the colors of 
the image elements (610) of the block i600)/The intensity codeword (720) is a 



representation of a set of mxiltiple 
intensity of the image elements (610) 



intensity modifiers for modifying the 
in : the block (600), Thq representation 



sequence (730) includes an intensity representation for each image element 
(610) in the block (600), where a representation identifies one of intensity 



modifiers in the intensity modijSer 



sei During decoding, the color and 



intensity codewords (710, 720) and intensity representations (730) are used to 



generate a decoded representation of . 
(600). 



the image elements (610) in the block 
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